184 research outputs found
Kernel Truncated Regression Representation for Robust Subspace Clustering
Subspace clustering aims to group data points into multiple clusters of which
each corresponds to one subspace. Most existing subspace clustering approaches
assume that input data lie on linear subspaces. In practice, however, this
assumption usually does not hold. To achieve nonlinear subspace clustering, we
propose a novel method, called kernel truncated regression representation. Our
method consists of the following four steps: 1) projecting the input data into
a hidden space, where each data point can be linearly represented by other data
points; 2) calculating the linear representation coefficients of the data
representations in the hidden space; 3) truncating the trivial coefficients to
achieve robustness and block-diagonality; and 4) executing the graph cutting
operation on the coefficient matrix by solving a graph Laplacian problem. Our
method has the advantages of a closed-form solution and the capacity of
clustering data points that lie on nonlinear subspaces. The first advantage
makes our method efficient in handling large-scale datasets, and the second one
enables the proposed method to conquer the nonlinear subspace clustering
challenge. Extensive experiments on six benchmarks demonstrate the
effectiveness and the efficiency of the proposed method in comparison with
current state-of-the-art approaches.Comment: 14 page
Decoupled Contrastive Multi-view Clustering with High-order Random Walks
In recent, some robust contrastive multi-view clustering (MvC) methods have
been proposed, which construct data pairs from neighborhoods to alleviate the
false negative issue, i.e., some intra-cluster samples are wrongly treated as
negative pairs. Although promising performance has been achieved by these
methods, the false negative issue is still far from addressed and the false
positive issue emerges because all in- and out-of-neighborhood samples are
simply treated as positive and negative, respectively. To address the issues,
we propose a novel robust method, dubbed decoupled contrastive multi-view
clustering with high-order random walks (DIVIDE). In brief, DIVIDE leverages
random walks to progressively identify data pairs in a global instead of local
manner. As a result, DIVIDE could identify in-neighborhood negatives and
out-of-neighborhood positives. Moreover, DIVIDE embraces a novel MvC
architecture to perform inter- and intra-view contrastive learning in different
embedding spaces, thus boosting clustering performance and embracing the
robustness against missing views. To verify the efficacy of DIVIDE, we carry
out extensive experiments on four benchmark datasets comparing with nine
state-of-the-art MvC methods in both complete and incomplete MvC settings
Image Clustering with External Guidance
The core of clustering is incorporating prior knowledge to construct
supervision signals. From classic k-means based on data compactness to recent
contrastive clustering guided by self-supervision, the evolution of clustering
methods intrinsically corresponds to the progression of supervision signals. At
present, substantial efforts have been devoted to mining internal supervision
signals from data. Nevertheless, the abundant external knowledge such as
semantic descriptions, which naturally conduces to clustering, is regrettably
overlooked. In this work, we propose leveraging external knowledge as a new
supervision signal to guide clustering, even though it seems irrelevant to the
given data. To implement and validate our idea, we design an externally guided
clustering method (Text-Aided Clustering, TAC), which leverages the textual
semantics of WordNet to facilitate image clustering. Specifically, TAC first
selects and retrieves WordNet nouns that best distinguish images to enhance the
feature discriminability. Then, to improve image clustering performance, TAC
collaborates text and image modalities by mutually distilling cross-modal
neighborhood information. Experiments demonstrate that TAC achieves
state-of-the-art performance on five widely used and three more challenging
image clustering benchmarks, including the full ImageNet-1K dataset
Contrastive Clustering
In this paper, we propose a one-stage online clustering method called
Contrastive Clustering (CC) which explicitly performs the instance- and
cluster-level contrastive learning. To be specific, for a given dataset, the
positive and negative instance pairs are constructed through data augmentations
and then projected into a feature space. Therein, the instance- and
cluster-level contrastive learning are respectively conducted in the row and
column space by maximizing the similarities of positive pairs while minimizing
those of negative ones. Our key observation is that the rows of the feature
matrix could be regarded as soft labels of instances, and accordingly the
columns could be further regarded as cluster representations. By simultaneously
optimizing the instance- and cluster-level contrastive loss, the model jointly
learns representations and cluster assignments in an end-to-end manner.
Extensive experimental results show that CC remarkably outperforms 17
competitive clustering methods on six challenging image benchmarks. In
particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100)
dataset, which is an up to 19\% (39\%) performance improvement compared with
the best baseline
An Enhanced Probabilistic LDA for Multi-Class Brain Computer Interface
There is a growing interest in the study of signal processing and machine learning methods, which may make the brain computer interface (BCI) a new communication channel. A variety of classification methods have been utilized to convert the brain information into control commands. However, most of the methods only produce uncalibrated values and uncertain results.In this study, we presented a probabilistic method "enhanced BLDA" (EBLDA) for multi-class motor imagery BCI, which utilized Bayesian linear discriminant analysis (BLDA) with probabilistic output to improve the classification performance. EBLDA builds a new classifier that enlarges training dataset by adding test samples with high probability. EBLDA is based on the hypothesis that unlabeled samples with high probability provide valuable information to enhance learning process and generate a classifier with refined decision boundaries. To investigate the performance of EBLDA, we first used carefully designed simulated datasets to study how EBLDA works. Then, we adopted a real BCI dataset for further evaluation. The current study shows that: 1) Probabilistic information can improve the performance of BCI for subjects with high kappa coefficient; 2) With supplementary training samples from the test samples of high probability, EBLDA is significantly better than BLDA in classification, especially for small training datasets, in which EBLDA can obtain a refined decision boundary by a shift of BLDA decision boundary with the support of the information from test samples.The proposed EBLDA could potentially reduce training effort. Therefore, it is valuable for us to realize an effective online BCI system, especially for multi-class BCI systems
Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification
Few-shot text classification aims to classify the text under the few-shot
scenario. Most of the previous methods adopt optimization-based meta learning
to obtain task distribution. However, due to the neglect of matching between
the few amount of samples and complicated models, as well as the distinction
between useful and useless task features, these methods suffer from the
overfitting issue. To address this issue, we propose a novel Adaptive
Meta-learner via Gradient Similarity (AMGS) method to improve the model
generalization ability to a new task. Specifically, the proposed AMGS
alleviates the overfitting based on two aspects: (i) acquiring the potential
semantic representation of samples and improving model generalization through
the self-supervised auxiliary task in the inner loop, (ii) leveraging the
adaptive meta-learner via gradient similarity to add constraints on the
gradient obtained by base-learner in the outer loop. Moreover, we make a
systematic analysis of the influence of regularization on the entire framework.
Experimental results on several benchmarks demonstrate that the proposed AMGS
consistently improves few-shot text classification performance compared with
the state-of-the-art optimization-based meta-learning approaches.Comment: COLING 202
Regulation of Irregular Neuronal Firing by Autaptic Transmission
The importance of self-feedback autaptic transmission in modulating
spike-time irregularity is still poorly understood. By using a biophysical
model that incorporates autaptic coupling, we here show that self-innervation
of neurons participates in the modulation of irregular neuronal firing,
primarily by regulating the occurrence frequency of burst firing. In
particular, we find that both excitatory and electrical autapses increase the
occurrence of burst firing, thus reducing neuronal firing regularity. In
contrast, inhibitory autapses suppress burst firing and therefore tend to
improve the regularity of neuronal firing. Importantly, we show that these
findings are independent of the firing properties of individual neurons, and as
such can be observed for neurons operating in different modes. Our results
provide an insightful mechanistic understanding of how different types of
autapses shape irregular firing at the single-neuron level, and they highlight
the functional importance of autaptic self-innervation in taming and modulating
neurodynamics.Comment: 27 pages, 8 figure
- …